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Assessing Vocational-Technical Training Needs 



INTRODUCTION 



This Guide is designed to assist countries in the Eastern Caribbean region in assessing 
their needs for vocational-technical training. Conducting needs assessment is part of the 
overall strategy of these countries for investing in vocational-technical training to provide 
the human resources needed for economic growth and improving the income of their 
citizens. 

The Guide is oriented to support decision-making in investments in occupational training. 
Thus, vocational education and training as discussed here includes education and training 
programs that provide trainees with skills and knowledge required for employment in 
specific occupations or a range of related occupations. 

Vocational education and training is distinguished here from education and training that 
is focused on imparting basic skills, such as reading comprehension and basic 
mathematics. While basic skills may be developed and improved during vocational 
education and training, basic skill development is not the primary purpose. 

Why Do Needs Assessment? 

Well-designed and competently delivered vocational-technical programs serve important 
purposes in any economy. They provide individuals with the skills and knowledge 
needed to succeed in the labor market and help meet employers’ needs for trained 
workers essential to business survival and expansion. By increasing the human capital 
available, appropriate vocational-technical programs also contribute to the overall 
development of the economy, growth in income, and the standard of living. 

Vocational -technical training can be expensive. Its development also takes time and 
leadership effort. Needs assessment is a very practical tool for achieving the purposes of 
vocational -technical training and using financial and leadership resources efficiently. 

Needs assessment helps ensure that training is related to the opportunities in the economy 
and is not wasted on training workers for occupational fields where jobs will not be 
available or that will not lead to economic self-sufficiency and economic growth. Needs 
assessment helps ensure that the individuals emerging from vocational-technical 
programs will have the skills and knowledge employers are seeking in their workers. 

Needs assessment results are used by those who make decisions about the specific types 
of training programs that will be offered and how training resources should be invested. 

What is Needs Assessment? 

Needs assessment is a defined process for identifying the type of training programs that 
should be offered to fulfill the purpose of vocational-technical training. The needs 
assessment should be based on a clear understanding of the role vocational-technical 
training is to play in the nation’s economy and educational system, and policy decisions 
on what are the driving forces for making program decisions. Decisions may be based on 
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meeting student demand, meeting short-term employer needs, or meeting longer-term 
economic goals. 

Needs assessment may include identifying occupations that are in demand in the labor 
market, that are appropriate for vocational-technical training, and that meet other criteria. 

The needs assessment may provide additional information about the characteristics of 
these occupations that is useful in developing training programs, such as employer hiring 
practices and entry-level requirements. Using this and other information, decision- 
makers can make informed choices about what training programs should be expanded or 
changed to meet the expected future demand. 

The results of the needs assessment will be an important part of the information used in 
decision-making about training, but it will not be the only information used. In market 
economies, it is also important for training decisions to be driven by student interest, and 
for student interest to be informed by clear, accurate, and accessible information about 
career opportunities. 

Needs assessment will not answer every question, however: 

■ Generally, needs assessment is not a substitute for a good labor market 
information system that addresses all segments of the economy. Needs 
assessments usually are focused on specific industry or occupation areas and are 
not designed to provide employment statistics on a regular production schedule. 
Ideally, a good labor market information system makes the needs assessment 
process easier, faster, cheaper, and more effective. 

■ Needs assessment, at least as described in this Guide, will not provide 
information on the specific skills and knowledges required in the occupation, 
information that is essential to develop the training curriculum. Additional 
activities are needed, particularly job analysis for the occupations identified 
during the needs assessment, using methods such as DACUM. 

■ Needs assessment does not provide information about specific job openings for 
use in job search and job placement activities. 

■ Needs assessment is not designed to provide career information for use by 
young people and their parents, or by adults, in making decisions about 
education, training, and careers. Needs assessment results can supplement other 
career information, however. 

In addition, other factors beside the results of the needs assessment will probably be 
taken into consideration in making vocational education and training program decisions. 
These may include the availability of teachers, facilities and equipment, binding 
resources, and a variety of other issues. 

Needs Assessment Within a Company 

This Guide discusses needs assessment from an economy-wide perspective. Training 
needs assessment may also take place within a company or organization. These 
assessments are conducted to help the company focus its training efforts and may address 
training needs to solve performance problems, training for newly hired employees, or 
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training to development of employees for advancement. Within-company needs 
assessments identify the required level and type of skills or knowledges, assess employee 
actual or expected (in the case of newly hired workers) skills and knowledges, and 
analyze the gap between the two sets of information. This gap is the training need. 

Needs assessment is followed by development, delivery, and evaluation of the training. 

While this Guide is not designed for primarily for assessments within companies or 
organizations, some of the same principles apply; Needs assessments within a company 
are discussed briefly in the final section of the Guide, and information on additional 
resources is provided. 
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OVERVIEW OF THE NEEDS ASSESSMENT PROCESS 



Conducting a successful needs assessment involves eight broad phases: 



1 . Laying the foundation by gaining authority and support for the project, including 
funding and staff resources. 

2. Establishing the policy framework for vocational-technical training by defining 
the goals and scope of the vocational-technical training system and its role in the 
nation’s economy and educational system. This includes how vocational- 
technical training relates to the country’s economic development strategy. 

3 . Engaging key stakeholders in the needs assessment process. 

4. Reviewing the economy by using existing economic and labor market data to 
identify what is already known about industry and occupational employment 
trends and other factors. 

5. Selecting training needs indicators for use in decision-making about vocational- 
technical programs, and determining whether the information is currently 
available. 

6. Gathering additional information, including determining the method for 
obtaining the information and actually collecting the information collection. 

This phase includes discussion with employers as well as gathering data. 

7. Analysis and presentation of the needs assessment results and recommendations 
to stakeholders and decision-makers for their use in deciding what types of 
training should be offered. Once these decisions are made, other activities (not 
part of the needs assessment) may take place, such as job analysis to identify the 
skills required and curriculum development. 

8. Putting the needs assessment results to work by developing and updating 
program curriculum, monitoring the labor market, providing career information, 
and collecting and analyzing program information. 



While these phases are generally sequential, some of them can be conducted at the same 
time. For example, Phase 4, Reviewing the Economy, can be done at the same time as 
other early phases in the process. 

Each of these phases is described in detail in the following sections of the Guide, 
including step-by-step procedures that can be adapted to local conditions. 



10 

Assessing Vocational-Technical Training Needs 



Page 5 



PHASE 1. LAYING THE FOUNDATION 



Before any needs assessment activity begins, it is important to lay the foundation for the 
overall effort. These initial efforts will ensure that the needs assessment has the 
appropriate support from its sponsoring organization, that expectations are clear, and that 
resources are available. At this stage, a communication strategy should also be 
considered. 



Laying the Foundation: Step by Step 



Step 1. Review this Guide. 

To attain an overall understanding of what the needs 
assessment requires, you should read this entire 
Guide at the outset of the project. 

Step 2. Solidify the Support of the Sponsor. 

Be sure the organization that is sponsoring the needs 
assessment - referred to throughout the Guide as the 
“Sponsor” - is firmly behind the project. The Sponsor may be a government ministry, a 
business organization, an educational institution, or some other organization. Regardless 
of the type of organization, the Sponsor must be committed to supporting the project and 
should have an interest in the use of the needs assessment results in making decisions 
about vocational-technical training. 

Part of solidifying Sponsor support is to clarify expectations. The Sponsor should have a 
clear and realistic understanding of 

■ What the needs assessment will provide, 

■ What the needs assessment will not provide, 

■ How long it is likely to take, 

■ Approximate cost of the project, and 

■ How finding will be obtained. 

Step 3. Develop the Project Plan. 

Prepare a written, broad plan for the overall project, including a general description of the 
needs assessment process to be used, who will be involved, and an approximate schedule. 
This plan should be updated at each phase of the needs assessment with additional detail 
as decisions are made. 



Laying the Foundation: 
Step by Step 

1. Review the Guide. 

2. Solidify Sponsor Support. 

3. Develop the Project Plan, 

4. Prepare a Project Budget 

5. Identify Project Staff. 

6. Develop a Communication 
Strategy, 
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Step 4. Prepare a Project Budget. 



Although exact costs cannot be identified at the outset, prepare an estimate of the cost of 
the needs assessment. The sources or potential sources of funding should be identified. 

The cost estimate should include the following types of costs: 

■ Staff, 

■ Stakeholder Group support, 

■ Conducting the review of the economy, 

■ Information collection activities, 

■ Analysis activities, 

■ Communications activities, and 

■ Dissemination of the needs assessment results, including printing or web 
dissemination 

The budget should identify actual or potential sources of funding and other resources. 
These may include the Sponsor, and private as well as public sources. 

TIPS: 

The resources provided to the project do not always have to be financial. For 
example, organizations such as universities, statistical agencies, or key businesses such as 
banks, may have the expertise to perform the review of the economy. If they understand 
the purpose and benefits of the needs assessment, they may be willing to perform the 
analysis as a nonfmancial contribution to the project. 

11 ^ 

Be sure the Sponsor agrees to the general range of costs, as well as the strategy for 
obtaining funding and other nonfmancial support. 

Step 5. Identify Project Staff. 

The needs assessment project will require staff support, including the following types of 
individuals; 

Project Leader. A single individual should be identified as the Project Leader or 
Project Director. This person will have overall responsibility for the planning 
and implementation of the project, maintaining communication with the 
Sponsor, organizing and supporting the Stakeholder Group, and supervising or 
working with staff and contractors. This individual should have the support of 
the Sponsor, as well as the following skills and knowledge: 

o Good communication skills, 
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o Understanding of the country’s vocational-technical education system, 

o Understanding of the roles of key stakeholders (ideally, the individual will 
be known by leaders in some of these groups), and 

o Project planning and implementation experience. 

■ Analyst. One or more individuals will be needed to gather and analyze 
information, and prepare the project report. In small-scale projects, the Project 
Leader may do some or all of these analysis tasks. The Analyst should have the 
following skills and knowledge: 

o Ability to identify and evaluate information sources, including 
communication with information providers, 

o Ability to organize and assemble information, 

o Ability to use basic information analysis techniques and, if appropriate, 
analytical software, 

o Knowledge of basic statistics and survey principles, 

o Knowledge of occupational classification principles, and 

o Good communication skills, especially written communication. 

■ Information collection staff. These individuals will be responsible for designing 
and conducting information collection activities, which may include employer 
surveys or qualitative methods. The specific skills and knowledge required will 
depend on the information collection method used. 

■ Administrative support staff. This person will provide clerical and other support 
to the project, as directed by the Project Leader. 

Step 6. Develop a Communication Strategy. 

The communication strategy should identify who needs to know about the project, and 
the purpose of communicating with them. For example, is the communication to make 
them aware of the project, or ask for their involvement in some way? The plan should 
also identify specific communication activities that will be conducted. Audiences for the 
communication activities might include: 

■ Key employer groups. Employers will likely be called upon to respond to 
information collection requests and are a key customer of vocational-technical 
training. The understanding and support of the project by key employer groups 
can help gain the cooperation of all employers in the information collection 
activities and acceptance of decisions based on the needs assessment results. 

■ Key education groups and leaders. Education leaders will be impacted by 
decisions based on the results of the needs assessment and therefore should be 
made aware of the project. 

■ Political leaders. These leaders will be called upon to act on funding and other 
aspects of the vocational-training system based on the results of the needs 
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assessment. They should be aware that a needs assessment is being conducted 
that will help guide these decisions. 

■ The economics statistics agency. This agency will be called upon for 
information and perhaps to provide the review of the economy. 

■ Other stakeholders^ Other groups and individuals may be identified who need 
to know about the project. 

■ General public. Information about the project, especially about its results, 
should be made available to the public. 

Examples of communication activities include: 

■ Meetings with and/or letters to the most important stakeholders. 

■ Developing a brochure explaining the purpose of the needs assessment project 
and who is involved. The brochure could be targeted to a specific audience, for 
example, employers. 

■ Distribution of the project report or results to targeted audiences. 

■ Press stories about the project and the results. 

■ Providing information on the web, such as a project description, questions and 
answers, and a contact for more information. 



TIP: 

It is important to have the Sponsor’s support for the communication strategy and 
approval of any communication materials that include the Sponsor’s name. 

The Stakeholder Group could be involved in developing the communication 
strategy. 



14 



Page 10 



Assessing Vocational-Technical Training Needs 



PHASE 2. ESTABLISHING THE POLICY FRAMEWORK 



In this phase, the policy context for the needs assessment is established. The broad goals 
and scope of vocational education and training are reviewed, and the role of vocational 
education and training in the educational system and its relationship to other parts of the 
system are considered. The relationship between vocational education and training and 
the country’s economic development strategy is also addressed. 

Vocational education and training can be designed to serve a number of different broad 
goals, usually set by national policy and often defined according to who has been 
identified as the primary client or “driver” of the system. 

Vocational education and training may be designed to respond to several types of 
“drivers”': 

■ Meeting Student Demand for Training. The vocational education and training 
system may prioritize responding to the choices of individual students. National 
policies may promote individual choice through enrollment and funding 
arrangements. Needs assessment in a student-choice driven system will focus 
on anticipating student demand and providing capacity to respond as demand 
changes over time, including adding or dropping courses and programs. Student 
choice should also be supported through career information and counseling 
services. Gasskov notes that in wealthier countries with student-demand driven 
systems, “pre-employment training is often seen as one of the avenues for the 
broad socialization of young people rather than a way of catering to labour 
markets.”^ 

■ Meeting Short-term Labor Market Demand for Trained Workers. In this 
approach, training is geared to respond to current signals of demand for 
particular skills and requests from employers for particular types of training. 

This demand-driven system will be responsive to immediate and particular 
needs and may have immediate payoff in terms of employment of trainees and 
productivity of enterprises. Care must be taken in interpreting labor market 
signals, however, so that demand for skills and skill shortages can be 
distinguished from unmet demands resulting from low wages or poor working 
conditions. Also, short-term demand driven training generally cannot be 
responsive to national strategic human resource development and economic 
development needs. 

■ Meeting Long-term Strategic Goals for Human Resource Development and/or 
Economic Development. In this approach, long-term trends in demand for 
human resources are examined and priorities for programs are established to 
meet these long-term needs. Long-term strategic planning is often aimed at 
supporting national economic development strategies and human resource 
development goals and developing vocational education and training capacity 
and responsiveness. Long-term planning is based on economic and 
demographic forecasts, as well as trends in student demand and the labor 
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market. It should be noted that forecasting the future is likely to be inaccurate, 
and decisions need to be reviewed regularly to take into account any 
fundamental changes in trends. 

The needs assessment process described in this Guide addresses these three goals. If 
other goals have been adopted, decisions will be needed about how the process should be 
changed as well. 



Establishing the Policy Framework: Step by Step 



Step 1. Review Vocational Education and 
Training Goals. 

It is critical that the goals be clearly 
established at the outset of the needs 
assessment process, as different goals require 
use of different assessment information and 
methods. 

Which of the three drivers of vocational- 
technical training are important for making 
program decisions: meeting student demand for training, meeting short-term labor 
market demand for trained workers, or meeting long-term strategic goals for human 
resource and/or economic development? Are there other policy goals that drive program 
choices? 

Step 2. Determine the Scope of Vocational Education and Training. 

For purposes of the needs assessment, the scope of vocational education and training 
needs to be defined. The needs assessment should focus on the level and type of training 
for which investment decisions will be made. Generally the scope should exclude short- 
term training, which usually is inexpensive and does not impart extensive skills, and thus 
is not a significant investment target for publicly supported training. The scope also will 
likely exclude education leading to bachelor’s level or higher degrees. The planning 
needs for higher education are probably being addressed elsewhere in the education 
system. Beyond these general guidelines, the decision on scope must be made based on 
policy considerations and the particular structure of education in the country. 

Step 3. Determine the Target Geographic Area. 

Determine the geographic area for the needs assessment. The area may be the country as 
a whole, some region within the country, or perhaps a local educational district. 
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Establishing the Policy Framework: i 

Step by Step | 

I 

1. Review vocational-technicai training !, 

goals. 

2. Determine scope. 

3. Determine target geographic area. 

4. Review economic development I 

strategy.: j 

5. Review education and training 
capacity. 



In making this decision, keep in mind that the labor market served by the educational 
system probably does not conform to political boundaries. Individuals cross boundaries 
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to seek work and often live in jurisdictions different from where they work. A useful 
concept to keep in mind is that a “labor market area” is a geographical area within which 
individuals seek work and employers find workers. In a practical sense, it is an area 
within which most workers live within commuting distance to their jobs. 

Step 4. Review the Country’s Economic Development Strategy. 

Decision-making about vocational-technical training will take place within the context of 
the country’s overall economic development strategy. The economic development 
strategy will likely identify labor market issues and include priorities or initiatives about 
the labor market or education. The strategy also may target certain sectors or industries 
for development efforts. 

The economic development strategy should be reviewed to identify issues and strategies 
that are relevant for vocational training needs. These may include strategies related to 
increasing income and strategies related to developing the education system. 

Step 5. Review Vocational Education and Training Capacity. 

The policy framework should include a review of the country’s education system and the 
role of vocational-technical education within the system. The structure and capacity of 
vocational education and training should be reviewed. How is the system organized? 

How many students can it accommodate with its current facilities and teaching staff? 
What programs does it currently offer and how many students are served by each 
program? What are the characteristics of the students, such as their age and gender? In 
what geographic areas are the programs offered? 

A written description of the system’s structure and capacity should be prepared for use by 
the Stakeholder Group (See Phase 3). 
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PHASE 3. ENGAGING KEY STAKEHOLDERS 



Probably the single most important step in conducting a successful needs assessment is to 
obtain the support and involvement of key stakeholders. These stakeholders are 
individuals and organizations who have a direct interest in successful vocational- 
technical training. They also include those who will use the needs assessment results in 
making decisions about what training will be offered, the level of resources to be 
provided, and how the resources will be allocated. 

Businesses, as employers of vocational-technical program students, are major 
stakeholders. Their involvement and cooperation are critical to the success of the needs 
assessment process. Businesses and officials responsible for economic development also 
have an interest in vocational-technical training because of its contribution toward human 
capital and increasing income levels. 

Those who finance and operate the training system are also important stakeholders, and 
their understanding and acceptance of the needs assessment results are critical. 
Organizations that represent the individuals who receive training also have a stake. 

These may be labor organizations (unions), community organizations, and representatives 
of students (for adult programs) and parents (for secondary programs). 

These stakeholders must be brought into the needs assessment process at the beginning 
through the creation of a “Stakeholder Group.” The role of this group must be carefully 
and explicitly defined at the outset of the needs assessment project. The Stakeholder 
Group may serve as advisors, providing suggestions and recommendations on the needs 
assessment process, or may have a more directive role in planning and managing the 
needs assessment. The appropriate role may depend on the amount of time and 
commitment stakeholders are willing to provide and the level of involvement needed to 
gain their acceptance of the needs assessment results. 

Stakeholders should have a continuing role after the initial needs assessment is 
completed, although the formal Stakeholder Group may be dissolved. Employers’ 
cooperation is necessary for the next step of identifying the skill aAd knowledge that are 
required in the demand occupations identified by the needs assessment (see Phase 8). 

The continuing support of all stakeholders will help generate the sustained effort needed 
to implement change. 



Engaging Key Stakeholders: Step by Step 

Step 1. Define the Role of the Stakeholder Group. 

Determine the answers to the following questions: 

■ What role should the Stakeholder Group play in the needs assessment project? 
Will they provide advice, or will they direct and oversee the project? 
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What activities will the Stakeholder Group perform? Potential activities 
include: 



o Conducting outreach to gain employer 
participation in data collection, 

o Making recommendations based on 
needs assessment results, and 

o Communications with the public and 
others about the project. 

What role, if any, will the St^eholder 
Group have in making decisions about 
vocational-technical program offerings? 



Engaging Key Stakeholders: 
Step by Step 

1. Define Stakeholder Group 
role. 

2. Identify Stakeholders. 

3. Solicit members. 

4. Arrange for ' care and 
feeding” of the Group. 

5. Convene Stakeholder 
Group, 

6. Say “Thank You.” 



Write a brief statement of the expected role of the 
Stakeholder Group in the needs assessment process. This statement should be used 
during the creation of the group, so potential members are aware of what they are being 
asked to do. The statement also should be reviewed by the Stakeholder Group at the 
beginning of its work. If the group itself is to help decide its role, its initial activity 
should be to develop a revised or new statement of their role. 



TIPS: 

Make sure the role of the Stakeholder Group is meaningful! You will be asking 
Stakeholder Group members for commitment of their time and energy, and they 
will want their effort to make a difference. If Stakeholder Group members do not feel 
they have a meaningful role, they may drift away from the effort or attack the results. 

Give the Stakeholder Group a formal name. This will help the group members 
identify with the project and make the project visible to employers, educators, and 
the public. Visibility is part of gaining acceptance of the needs assessment results. 

Do not expect the Stakeholder Group members to do the work of performing the 
needs assessment. This is the job of the staff 



Step 2. Identify the Stakeholders. 

A. Identify key organizations that should be represented, such as the following: 

■ Businesses: Chambers of Commerce, business associations, key employers in 
the economy, 

■ Education leaders: national ministry level, institution level, 

■ Labor organizations: unions, 

■ Economic development: those responsible for implementing national strategy, 

■ Community organizations, and 
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Parent or student organizations. 



B. Determine the desired level of the individual representatives on the Stakeholder 
Group, such as: 

■ The level of business representatives, for example, chief executives, human 
resource directors, plant managers, supervisors, and 

■ The level of education leaders, for example, rank within national ministry, 
school superintendent, or teacher. 

TIP: 

All Stakeholder Group members do not have to be at the same level. A mix of 
levels is appropriate, such as including chief business executives and teachers on 
the same group, as long as custom and the way the group conducts its business encourage 
participation by all members. It is best not include superiors and subordinates from the 
same organization, however, as this may inhibit discussion. 



C. Determine if any existing organizations can serve as the Stakeholder Group, for 
example, a national training board that includes all of the identified stakeholders. 

■ If existing groups are identified, determine whether and how to approach them 
about assuming the role of the Stakeholder Group. 

■ If the decision is not to approach existing groups, determine what 
communication with the existing groups about the creation of the Stakeholder 
Group is appropriate. The support of these existing groups may be important 
when decisions are made based on the needs assessment results. 

Step 3. Solicit Individuals to Join the Stakeholder Group. 

A. Be sure the group represents all important segments of the economy/population, 
such as: 

■ Different geographic areas, 

■ Different ethnic or race groups, 

■ Female and male members, and 

■ Major industries in the countries economy. 

B. Decide on the number of members desired. Keep the group to a workable size, 
with at least 10 members but ideally no more than 15 to 20. 
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C. Issue invitations to prospective members. 

■ Choose the way of issuing the invitation that best fits your culture and political 
environment, for example, formal invitation or an informal invitation. 

■ Decide on who should issue the invitation. Should it be from a high-level 
official? From a business leader? 

■ Be sure the invitation is clear on who the Sponsor is, the expected role of the 
Stakeholder Group, the commitment of time that is expected, and how long the 
group is expected to be active. 

■ Indicate whether expenses for participation will be paid for. 

TIP: 

If you are seeking the participation of business chief executives and other high- 
level individuals, the invitation should come from a high-ranking source. 



Step 4. Arrange for the ’’care and feeding” of the Stakeholder Group. 

Effective Stakeholder Groups need support, leadership, and attention to meeting logistics. 

■ Staffing. Be sure the group members know who will provide staff support and 
what the staff will do for the group. 

■ Group leadership. Determine how the group will be led. Will there be a 
chairperson? If so, will the chairperson be selected by the group or by the 
Sponsor? Group leadership and decision-making processes may depend on the 
custom in your country. 

■ Meeting planning. Staff and the group leadership should work together on 
setting schedules for the group’s meetings and other activities, establishing 
meeting agendas, identifying materials needed either prior to or during meetings, 
and deciding how meeting activities will be recorded (for example, meeting 
notes or minutes). 

■ Meeting logistics. Staff should ensure that appropriate meeting space is 
provided, along with any equipment such as audio/visual equipment, and that 
any refreshments or meals are provided as needed. Staff should also arrange for 
any other meeting resources, for example, outside presenters or facility tours. 

■ Public recognition. The Sponsor, staff, and group leadership should decide 
whether public announcements of the creation of the Stakeholder Group are 
appropriate and how to communicate with political leadership, the public, the 
business and education community, and others on the role and activities of the 
Stakeholder Group. 

TIP: 

If there is to be a group chairperson, be sure the Stakeholder Group members know 
how the chairperson is selected and what the chairperson’s role will be. 
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Step 5. Convene the Stakeholder Group. 



Once the Stakeholder Group is established, it should begin its work promptly. The 
Stakeholder Group’s work will involve several phases: getting started, beginning the 
work, conducting the needs assessment, and wrapping up. 

A. Getting Started. 

The following activities will develop common understanding among group members of 
their role and purpose and provide them with a shared foundation of information 
important to the needs assessment process. These activities should be carried out at the 
beginning of the Stakeholder Group’s work: 

■ Introductions. At the first meeting, begin with introductions so all members and 
the staff are identified. Provide a list of group members and their organizational 
affiliation and names and contact information of staff Name tags or name tents 
on the meeting table can also help members get acquainted. 

■ Role clarification. At the first meeting, review and discuss the written statement 
of the Stakeholder Group’s role and responsibilities. If the group itself is to help 
decide its role, its initial activity should be to develop a revised or new statement 
of their role. Revisit the role statement as necessary throughout the life of the 
Stakeholder Group. 

■ Leadership. If the group is to select its oAvn chairperson, selection should be 
done at the first meeting. 

■ Ground rules. Depending on the custom in your country, ground rules for how 
the Stakeholder Group will conduct its business may be discussed and put in 
writing. This will help avoid conflict later on. 

■ Discuss what needs assessment is, its purpose, and how the results will be used 
in making decisions about vocational-technical training programs. Be sure these 
points are clear and that all group members understand them. 

■ Introduce the staff who will provide Stakeholder Group support. Provide 
information Stakeholder Group members need about logistics, such as plans for 
meeting locations and how to get paid for their expenses (if payment is being 
provided). 

■ Review the needs assessment process. 

■ Clarify any questions about the purpose, scope, and timing of the needs 
assessment project. 

■ Identify the points on which Stakeholder Group input, advice, recommendations, 
and possibly decisions (depending on the role of the group) are to be offered. 
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Review the definition of vocational education and training and provide 
information on the current organization and scope of vocational-technical 
training. 



TIP: 



Ask a representative of the Sponsor to attend the opening of the first meeting. If 
the Sponsor has selected the chairperson or other group leadership, the Sponsor 
should introduce the chairperson/leader(s). The Sponsor can also express appreciation 
for Stakeholder Group members’ participation and discuss Sponsor expectations of the 
Group. 



B. Beginning the Work. 

The following activities start the real work of the Stakeholder Group, and should begin 
quickly after the start-up activities are completed. This stage of the work should result in 
decisions about how the needs assessment project will be carried out: 

■ Present the Economic Review and discuss its implications for vocational- 
technical training. (See Phase 4.) 

■ Present economic development strategy and discuss its implications for 
vocational-technical training. 

■ Provide advice or decisions on the scope of the needs assessment, including 
discussion on how the information should be used in decision-making. 

■ Provide advice or decisions on how the needs assessment will be conducted. 

■ Determine how to encourage employer participation in information collection 
and discussion to assure representative results. 

C. Conducting the needs assessment. 

Once decisions have been made on how the needs assessment will be conducted, the 
assessment should begin. As the assessment is being carried out, the burden of the work 
is primarily on the staff The Stakeholder Group may have the following types of 
activities: 

■ Participate in publicity or communications activities, as identified in the project 
plan, especially employer outreach. 

■ Receive progress reports fi-om the staff on implementation of the needs 
assessment procedures. 

■ Provide guidance or assistance as the needs assessment is conducted, including 
how to deal with issues such as low employer participation. 
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D. Wrapping up. 

The following activities should be carried out as the end of the Stakeholder Group’s work 
nears: 

■ Receive and discuss the needs assessment results and the analysis of the results 
prepared by the staff 

■ Decide on any recommendations that stem from the needs assessment results. 

■ Review and approve the final needs assessment report. 

■ Transmit the final needs assessment report to the Sponsor. 

Step 6. Say “Thank You.” 

At the conclusion of the Stakeholder Group’s work, it is appropriate and important to 
provide the members with recognition and appreciation for their effort and contributions. 
The form of recognition will vary depending on customs in your country but could 
include: 

■ Letter from the Sponsor expressing appreciation. 

■ Presentation of a certificate or other token of recognition, perhaps at a final 
meeting or at a public event. 

■ Including Stakeholder Group member names in the final needs assessment 
report. 

■ Including Stakeholder Group member names in any publicity about the results 
of the needs assessment. 



TIP: 






Do not neglect this step! You want the Stakeholder Group members to end their 
work feeling valued. Remember, you may be asking them to participate in future 
projects! This step also clearly marks, in a graceful way, the end of the Stakeholder 
Group’s existence. 
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PHASE 4. REVIEWING THE ECONOMY 



Purpose of the Economic Review 

The Economic Review provides an understanding of what factors and trends are driving 
the country’s labor market, which is the essential context for developing a demand-driven 
vocational -technical training system. An understanding of the structure and trends of the 
country’s population, output, income, labor force, industries, and occupations is 
fundamental to identifying needs for skills. This information will also guide the 
decisions about industries or occupations that might be targeted during the needs 
assessment. 

To achieve this purpose, the Economic Review should be presented with the needs 
assessment audience in mind: the audience members are likely not economists and may 
not be familiar with all of the relevant economic concepts and data. The Economic 
Review report and the presentation of the Economic Review results should be brief and 
clear and should avoid technical jargon. 

Ideally, the Economic Review should be conducted by an individual who has a 
background in economics and knowledge of the available information sources and 
economic analysis methods, as well as familiarity with the country’s economy. This 
person should bring objectivity as well as knowledge to the task. Objectivity means that 
the individual will describe what is known about the various topics and will not perform 
the review with the intent of advocating a particular policy decision or conveying a 
biased view of the economic situation. 

Phase 4 can be conducted at the same time Phases 2 and 3 of the needs assessment ^e 
underway. Scheduling the work on the Economic Review in this way should ensure that 
the Economic Review is completed for presentation to the Stakeholder Group at the 
beginning of their work, as discussed in Phase 3, Step 5. 

Content of Economic Review 

A comprehensive Economic Review should cover the following topics. In many cases, 
information about all of these topics is not available or may not be of sufficient quality or 
detail to fully depict the economic conditions. In such instances, the review should cover 
as much as possible without distortion. 

A. Population and Labor Force 



Population. The Economic Review should begin with an examination of 
population size and trends, to provide an understanding of the general supply of 
workers expected in the future. If population forecasts are available, they should 
be examined to identify the expected size and rate of growth for the number of 
working age individuals, who will comprise the future labor force of the 
country. Within the working age population, it is important to examine the age 



er|c 



Assessing Vocational-Technical Training Needs 



25 



Page 23 



structure: what portion of the population is expected to reach retirement age? 
What portion will be in the prime working years? What portion will be younger 
inexperienced workers? How many young people will be entering the 
educational system? 

Labor Force Size and Trends. Similar to the examination of population, the size 
and growth of the labor force and labor force participation rates should be 
described. Factors that are impacting labor force growth should be identified. 

Labor Force Composition. The composition of the labor force by age, gender, 
and other factors important in the country (race or ethnicity, for example) should 
be described, including any recent changes. If population forecasts are 
available, it may be possible to estimate the size and of the future labor force 
and its composition by age and gender.^ The labor force comprises the expected 
future labor supply. 



Broad Economic Conditions and Trends 

Gross Domestic Product (GDP) Size and Trends. GDP information indicates 
the overall level of production and growth in the country’s economy. Trends in 
GDP per capita or GDP per worker are particularly usefiil, as they indicate the 
returns to human capital and progress in improving the standard of living. 
Economic development strategies, including strategies to increase human capital 
through education, may be aimed at increasing GDP per capita. 

Income. Trends in income of individuals or households are also useful 
indicators of progress in improving the standard of living. 

GDP by Sector. Understanding the demand side of the labor market begins with 
understanding the industries present in the economy and trends in output of 
products and services. Output by major industrial sector (agriculture, 
manufacturing, etc.), as well as GDP per worker, indicates the contribution of 
various sectors to economic growth. This information will be useful in the 
needs assessment if decisions are needed to target training toward high growth 
or high output sectors. 

Import/Export Trends. Information on imports and exports, especially by type 
of product, indicates the country’s links with the external markets and may 
indicate the direction of growth. 



Employment Conditions and Trends 

Total Employment Level and Trends. Growth or decline in GDP will result in 
changes in the demand for workers, measured by the level and trends in total 
employment. The examination of employment trends should capture trends by 
gender and other key demographic groups. 

Employment and Trends by Industry Sector. Review the employment level and 
trends in the country’s important industry sectors. Ideally, the industry analysis 
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will include development of employment forecasts, which requires historical 
employment data as well as expertise in economic forecast methods. 

■ Employment and Trends by Occupation Group. Employment by occupation 
indicates the overall demand for workers with different types of skills. If data 
are available, the Economic Review should describe the level of and trends in 
employment by occupation group, including forecasts of future employment. 



D. Education Conditions. Education information indicates the overall level of 
human capital available in the economy. The overall level of education of the population, 
and how the education level may be changing, should be described. Education level and 
trends for groups within the population, such as for different age groups and for men and 
women, should be shown. 



Reviewing the Economy: Step by Step 

Step 1. Identify and Obtain the Available Data. 

For each part of the Economic Review’s content, identify the available data sources, 
including information about definitions and concepts, and documentation of the data 
sources. Obtain the actual data, preferably in electronic format. 

Step 2. Analyze the Data. 

Analyze the available data to identify significant points regarding the structure and trends 
relevant to each part of the Economic Review (population, GDP, employment, etc.). 
Develop a brief descriptive narrative and charts and simple tables that depict the most 
important information. 

Step 3. Identify Implications for Vocational-Technical Training. 

Examples of implications might include answers to the following questions; 

■ How are population trends impacting 
current and future education enrollments? 

■ Does the economic development strategy 
target particular industries or geographic 
areas, for which vocational -technical 
training needs may be targeted during the 
needs assessment? 

■ What groups within the population are not 
fully participating in the economy because 
of their lack of education and training or 

BEST COPY AVAILABLE 
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other reasons (for example, low labor force participation of women)? 

What industries are contributing the most to the country’ s growth and income 
potential? 

How are occupational trends affecting the level of education and skill 
requirements? 



Step 4. Prepare the Economic Review Report. 

Write a report containing the results of the review. Keep in mind that many readers of 
the report will not be economists or necessarily be conversant with some of the basic 
concepts used. Clear presentation, using non-technical language, is essential. The report 
should also be brief, and not require an extensive time commitment on the part of the 
reader. 

TIPS 

Use charts and clear, brief tables to illustrate important points. Be sure the text also 
makes the same points, however, as some readers understand text more easily while 
others prefer graphics. 

Provide a brief glossary of essential terms that a general reader may not know. 

Write background notes about all information sources! Documentation is part of 
maintaining objectivity, as it lets readers know where they can find the information. 
Keep in mind that needs assessment is a continuing process. You may need to update the 
Economic Review in the future, and documenting the information sources will make it 
easier the next time. 

Be sure to mention major limitations of the data if they are likely to impact the 
conclusions drawn from the data about vocational-technical education. 

Step 5. Present the Economic Review. 

The Economic Review will be presented to the Stakeholder Group and perhaps to other 
audiences. As with the written report, the presentation should be made with the non- 
technical nature of the audience in mind. 
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PHASE 5. SELECTING TRAINING NEEDS INDICATORS 



In this phase of the needs assessment, the specific information requirements are 
determined and data availability is determined. Decisions are also made about what 
additional data need to be gathered. Methods for gathering the additional data are 
discussed in Phase 6. 

A Note about Industries and Occupations 

The distinction between “occupations” and “industries” is fundamental to the needs 
assessment. 

“Occupation” refers to the nature of the work performed by individual workers. 

Typically, occupations are defined according to the tasks performed by the worker. 
Training imparts skills to individuals to prepare them to perform work in various 
occupations. Examples of occupations are carpenter, teacher, and hotel front desk clerk. 

By contrast, “industry” refers to the type of product or service produced by the employing 
organization, or the activity conducted by the organization. Examples of industries 
include construction, education services, and hotels. 

Jobs in a specific occupation may be found in many industries, or in only one or a few 
industries. Carpenters, for example, are employed primarily in the construction industry, 
but are also found in other industries such as education and hotels, where there is a need 
to repair and maintain buildings and other wood structures. 



Classification Systems 

In the needs assessment, information about occupations and industries should be based on 
an established classification system, preferably a system that is used in the country’s 
economic and statistics data systems. 

Classification systems usually consist of titles and definitions of specific industries or 
occupations, and a hierarchical coding system. For industries, the definitions may 
describe the types of products or services produced and perhaps the production methods 
used. For occupations, definitions usually describe the work being performed and may 
list tasks and related job titles. 

Use of an established classification system will eliminate the additional effort of 
developing occupational or industry categories for use in the needs assessment, and will 
allow users to refer to definitions that provide additional detail about the nature of the 
industries or occupations. Using recognized classification systems also allows 
comparison of the needs assessment results with other available data, making analysis 
and updates easier. 

If the country has not adopted a specific classification system, other existing 
classification systems may be used. ^ For occupations, the International Labor 
Organization’s 1988 International Standard Classification of Occupations (ISCO-88) is 
the most widely adopted system. Although it is rather dated, the ISCO may still be 
appropriate for developing economies. More recent systems are the U.S. Stcmdard 




Assessing Vocational-Technical Training Needs 



29 



Page 27 



Occupational Classification, and Canada’s National Occupational Classification, both 
adopted in 2000. 

For industries, the United Nations Statistics Division maintains the International 
Standard Industrial Classification (ISIC), last updated in 1989. The United States, 
Canada, and Mexico have adopted the North American Industrial Classification System 
(NAICS), which classifies businesses based on the activities in which they are primarily 
engaged. In the U.S., NAICS is replacing the 1987 Standard Industrial Classification 
(SIC). 

To manage information for labor market analysis, it will also be very useful to have a 
classification structure for education and training programs. The United Nations 
Statistics Division also provides the International Standard Classification of Education 
(ISCED) issued in 1997. In the U.S., the 2000 Classification of Instructional Programs 
serves this purpose. 

Sources for the classifications mentioned above are listed in the References section of the 
Guide. 



Relating Training Programs and Occupations 

Labor market analysis requires that the relationships between education and training 
programs and occupations be identified. Programs may prepare individuals for a single 
occupation or, more often, several occupations. Conversely, preparation for some 
occupations is through a single type of program, while other occupations can be entered 
through any of several different types of training programs. 

The relationship between a program classification and an occupation classification is 
often referred to as a “crosswalk.”^ A crosswalk is a database that can be used to present 
the information organized either by program or occupation. 

For example in Figure 1, crosswalk information for nursing occupations is show, listed in 
occupation order. Note that for the occupation 32502, Registered Nurses, one program 
category is shown, 5 1. 1601, Nursing (RN Training). This program is related to the 
Registered Nurse occupation at the Bachelors through Doctoral levels. For occupation 
32505, Licensed Practical Nurses, four programs are listed: 51. 1601, Nursing (RN 
Training) at the Associate levels and below; 51.1613, Practical Nurse (LPN Training); 
51.1614, Pre-Nursing; and 51.1699, Other Nursing. 



Figure 1. Occupation-Program Crosswalk for Nursing Occupations, listed in Occupation Order 


Occupation 


Program 


Code 


Title 


Code 


Title 


Level 


32502 


Registered Nurses 


51.1601 


Nursing (RN Training) 


Doctoral 


32502 


Registered Nurses 


51.1601 


Nursing (RN Training) 


Post Master's Certificate 


32502 


Registered Nurses 


51.1601 


Nursing (RN Training) 


Masters 


32502 


Registered Nurses 


51.1601 


Nursing (RN Training) 


Post Baccalaureate Certificate 
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32502 


Registered Nurses 


51.1601 


Nursing (RN Training) 


Bachelors 


32505 


Licensed Practical Nurses 


51.1601 


Nursing (RN Training) 


Associate 


32505 


Licensed Practical Nurses 


51.1601 


Nursing (RN Training) 


At least one- but less than two- 
year award 


32505 


Licensed Practical Nurses 


51.1601 


Nursing (RN Training) 


Less than one-year award 


32505 


Licensed Practical Nurses 


51.1613 


Practical Nurse (LPN Training) 


Adult PT 


32505 


Licensed Practical Nurses 


51.1613 


Practical Nurse (LPN Training) 


Postsecondary FT 


32505 


Licensed Practical Nurses 


51.1613 


Practical Nurse (LPN Training) 


Secondary 


32505 


Licensed Practical Nurses 


51.1614 


Pre-Nursing 


Secondary 


32505 


Licensed Practical Nurses 


51.1699 


Other Nursing 


Adult PT 


32505 


Licensed Practical Nurses 


51.1699 


Other Nursing 


Postsecondary FT 



In Figure 2, crosswalk information is presented for nursing programs. For program 
5 1 . 1601, Nursing (RN Training) is related to three occupations. At the bachelors and 
masters level, this program is related to occupation 15008, Medicine and Health Service 
Managers, which includes people who work as nursing directors. The program is also 
related to 32502, Registered Nurses, and 32505, Licensed Practical Nurses. Finally, at 
levels of less that two years of training, the program is related to occupation 66026, 
Pharmacy Aides. 



Figure 2. Occupation-Program Crosswalk for Nursing Programs, listed in Program Order 


Occupation 


Program 


Code 


Title 


Code 


Title 


Level 


15008 


Medicine and Health Services 
Managers 


51.1601 


Nursing (RN Training) 


Bachelors 


15008 


Medicine and Health Services 
Managers 


51.1601 


Nursing (RN Training) 


Masters 


32502 


Registered Nurses 


51.1601 


Nursing (RN Training) 


Doctoral 


32502 


Registered Nurses 


51.1601 


Nursing (RN Training) 


Post Master's Certificate 


32502 


Registered Nurses 


51.1601 


Nursing (RN Training) 


Masters 


32502 


Registered Nurses 


51.1601 


Nursing (RN Training) 


Post Baccalaureate Certificate 


32502 


Registered Nurses 


51.1601 


Nursing (RN Training) 


Bachelors 


32505 


Licensed Practical Nurses 


51.1601 


Nursing (RN Training) 


Associate 


32505 


Licensed Practical Nurses 


51.1601 


Nursing (RN Training) 


At least one- but less than 
two-year award 


32505 


Licensed Practical Nurses 


51.1601 


Nursing (RN Training) 


Less than one-year award 


66026 


Pharmacy Aides 


51.1601 


Nursing (RN Training) 


At least one- but less than 
two-year award 


66026 


Pharmacy Aides 


51.1601 


Nursing (RN Training) 


Less than one-year award 


32505 


Licensed Practical Nurses 


51.1613 


Practical Nurse (LPN Training) 


Adult PT 


32505 


Licensed Practical Nurses 


51.1613 


Practical Nurse (LPN Training) 


Postsecondary FT 


32505 


Licensed Practical Nurses 


51.1613 


Practical Nurse (LPN Training) 


Secondary 


32505 


Licensed Practical Nurses 


51.1614 


Pre-Nursing 


Secondary 


32505 


Licensed Practical Nurses 


51.1699 


Other Nursing 


Adult PT 


32505 


Licensed Practical Nurses 


51.1699 


Other Nursing 


Postsecondary FT 
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Occupations and Skills 

More and more attention is being paid in vocational education and training to discussing 
“skill requirements” in the labor market rather than “occupations.” ® Skills information 
has always been critical for developing training curricula and usually has been obtained 
through job analysis and employer input. 

The new focus on skill requirements stems from a number of considerations, including 
. recognition that occupational classifications are rather rigid and hard to keep up to date, 
and that even within occupations with stable definitions, the skill requirements of the 
work may be changing rapidly. Also, labor markets are becoming increasingly fluid and 
hard to predict. Employers are more often looking for workers with certain skills rather 
than specific occupational training or qualifications, and employment forecasts based on 
past trends may not be good indicators of the future. 

In discussing skills, it is useful to refer to a hierarchy of skills, ranging from basic or 
fundamental skills to specialized skills. The Occupational Information Network 
(0*NET) from the United States provides one such hierarchy. 0*NET is built around a 
“content model”^ that includes three skill levels; 

■ Basic Skills- skills that facilitate learning and information acquisition. Examples 
include: 

o Content Skills - fundamental skills needed to work with or acquire more 
specific skills, such as reading comprehension and active listening. 

o Process Skills - procedures that contribute to the more rapid acquisition of 
knowledge and skill, such as critical thinking and active learning 

■ Cross-Functional Skills — skills that facilitate performance of activities that 
occur across jobs, such as social skills and complex problem-solving skills. 
Examples include: 

o Technical Skills- skills used to design, set up, operate, and correct 

malfunctions involving application of machines or technological systems, 
such as equipment selection and testing. 

o Systems Skills - skills used to understand, monitor and improve socio- 
technical systems, such as systems perception and judgment and decision- 
making. 

o Resource Management Skills- skills used to allocate resources efficiently, 
such as management of time and financial resources. 

■ Occupation-Specific Skills - skills for performance in specific occupations. 



In needs assessment, discussions of skill requirements often are concerned with basic and 
cross-functional skills (or their counterparts in other skills hierarchies). ® Information 
on these skill requirements is useful for ensuring that vocational education and training 
programs and other parts of the educational system are teaching the needed levels of 
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basic and cross-functional skills. Information on occupation-specific skills is needed for 
curriculum development in vocational education and training programs, however. Ways 
of identifying these skills are discussed in Phase 8. 



Employment Forecasts 

Forecasts of employment trends for industries and occupations are important training 
needs indicators, especially when long-term strategic goals are being addressed. Other 
forecasts, such population, labor force and school enrollment, are also useful. 

While the development of such forecasts has been underway for several decades in 
industrial countries, they remain a topic of some controversy. ^ Forecasts are by 
definition conjectures (sometimes based on very sophisticated analysis) about future 
trends. Forecasts therefore may be inaccurate. Use of simplistic methods or poor quality 
data can add to their inaccuracy. Also, forecasts may be quickly outdated by major 
unexpected events. Consequently, care must be exercised in interpreting forecasts, and 
they should be used along with other indicators in decision-making. 

Forecasts generally are prepared one or more techniques: 

■ Economic or demographic models provide quantitative estimates of population, 
labor force, employment and job openings for some future date based on models 
of population change and economic growth. These models may be quite 
sophisticated or may involve simple extrapolations and usually require extensive 
data resources. 

■ Group processes, especially the Delphi technique, are used to forecast future 
trends and provide qualitative information on the participants’ expectations of 
broad trends. See Phase 6 for a description of the Delphi technique. 

■ Employer surveys, although largely discredited as a method for producing 
quantitative forecasts, continue to be used. This approach is based on the 
seemingly obvious premise that employers know their future needs and are 
willing to share them. This method, however, lacks any firm theoretical 
foundation and does not assure that responding employers are using the same 
overall assumptions about economic conditions. Employers also may be 
unwilling to provide forecasts that reveal their plans regarding plant re- 
locations, expansions, or contractions. When used with care, employer surveys 
may provide useful results in the near-term if they focus on identifying factors 
that are driving change. 



Types of Training Needs Indicators 

The specific information used in needs assessment may be thought of as a set of 
indicators, that is, measures that represent some aspect of the need for vocational 
education and training. We will focus on four types of indicators: 
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Overall Labor Market Indicators-, measures of the overall structure and growth 
of the population, the economy, and the labor force, without regard to specific 
occupations. 

Occupational Demand Indicators-, measures of the need for new workers with 
the skills required, including basic and cross-functional skills as well as 
occupation-specific skills. 

Student Training Demand Indicators-, measures of students demand for training 
in general and for training in specific programs. 

Training Supply Indicators: measures of the capacity of the training programs 
and educational system to meet the demand for training. 



Generally, it is important to use several indicators when making decisions. It is not 
unusual for indicators to point in different directions, so using several indicators may help 
clarify the situation. Also, understanding of the accuracy and definition of the data is 
important when interpreting the indicators. For high-stakes decisions, as many indicators 
for which sound information can be obtained should be used. 

Developing the indicators involves obtaining and analyzing various demographic, labor 
market, and education information. The analysis is most successful when a 
comprehensive and objective employment statistics system is available that gathers and 
disseminates data on a regular basis using sound statistical and data collection methods. 

While a comprehensive labor market and education statistics systems may not be 
available, all Eastern Caribbean countries have some basic information. Additional 
information will likely need to be collected during the needs assessment, as described in 
Phase 6. 



Different Indicators for Different Decision Drivers 

In Phase 2, we discussed the broad goals of vocational education and training and the 
three alternative drivers of decision-making: 

■ Meeting Student Demand for Training, 

■ Meeting Short-term Labor Market Demand, and 

■ Meeting Long-term Strategic Goals. 

Because these drivers determine the nature of the planning decisions made, they also 
impact the type of training demand indicators that should be selected for the needs 
assessment. For each alternative, different types of information are needed. 
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Table 1 depicts, in general, how the four types of indicators relate to the decision drivers. 
For example, overall labor market demand indicators are generally not relevant in 
decision-making related to meeting student demand, but are helpflil where meeting short- 
term labor market demand is the priority. Overall labor market demand indicators are 
critical to planning related to long-term strategic goals, however. 

These general relationships are discussed in more detail below. There are some 
exceptions to the general relationships shown in Table 1, which will be apparent as 
specific indicators are examined later in this chapter. 

Overall Labor Market Indicators. Indicators of overall economic and labor market 
conditions and trends are helpful when decisions are driven by short-term labor market 
demand for training, and critical when long-term strategic needs are the driver. In a 
student-demand driven environment, this information is generally not relevant, although 
it should underlie any career information provided to students. 

Student Training Demand Indicators. In a student demand-driven environment, needs 
assessment is a process of identifying current and anticipated student demand for 
training, especially demand for specific types of occupational training. The available 
training capacity and the extent to which existing training programs are meeting student 
demand are also evaluated. 

Ideally, the needs assessment should compare student demand with labor market demand 
for occupational skills provided by the training. This comparison suggests how well 
students’ training choices reflect the labor market. Significant mismatches between 
student choices and labor market demand can be addressed by improving the information 
available to students in making their decisions, such as career information, career 
awareness programs, and job search information. Student recruitment may also be 
needed for training programs that provide good labor market opportunities. 

Short-term Labor Market Training Demand Indicators. Short-term labor market demand 
for training can be assessed primarily by analyzing various occupational demand 
indicators, such as the number of job openings and employer hiring activity. 

Care should be taken to evaluate the nature of shortages or “hard-to-fill” vacancies. 



Table 1. Relating 


I'pes of Indicators to Decision Drivers 


Type of Indicator 


Decision Drivers 


Student Demand 


Short-Term Labor 
Market Demand 


Long-Term 
Strategic Goals 


Overall Labor 
Market Indicators 


Generally not 
relevant 


Helpful 


Critical 


Occupational 
Demand Indicators 


Helpful 


Critical 


Critical 


Student Demand 
Indicators 


Critical 


Generally not 
relevant 


Generally not 
relevant 


Training Supply 
Indicators 


Critical 


Critical 


Critical 



ERIC 
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especially the wages and working conditions being offered. Low wages or poor working 
conditions, rather than lack of training, may be the cause of short-term shortages of 
workers. Short-term shortages may also result from the time lag between identifying 
training needs and the completion of training programs by students. 

Student training demand indicators are also useful in assessing short-term training needs. 
Training needs may not be met if student interest in the training programs is insufficient. 
Training Supply indicators should also be examined to determine whether training 
capacity is available or needs to be expanded to meet student demand. 

Long-Term Strategic Training Needs Indicators. In planning to meet long-term strategic 
goals, such as human resource development or economic development goals, the needs 
assessment is directed primarily to identifying long-term occupational demand rather than 
short-term, immediate employer needs. 

Economic development goals and policies should be explicitly recognized, especially if a 
policy decision was made in Phase 2 that the training system will be directed toward 
helping meet these development goals. 

Occupation demand indicators are among the data needed, especially industry and 
occupation employment forecasts, and estimates of job openings resulting from 
employment growth and from the need to replace workers who leave the labor force. 

Specific Indicators 

Table 2 lists 33 specific indicators, listed according to the type of indicator. The first 
column of this table provides a sequence number for easy reference to each indicator, and 
the second column shows the name of the indicator. The next three columns show the 
relevant decision drivers for the indicator. For example, indicator number 3, labor force 
trends and forecasts, is relevant to decisions about short-term labor market demand and 
strategic goals. The remaining columns discuss how each indicator may be used in needs 
assessment, provide additional comments, and note the general sources of information for 
the indicator. 

Note that at the end of Table 2, indicator 29, Number of completions by program, and 
indicator 31, Number of students enrolled by program, appear twice, once under “Student 
Training Demand” and once under “Training Supply”. These indicators can be used in 
both categories. 



36 



Page 34 



Assessing Vocational-Technical Training Needs 



Table 2. Specific Needs Assessment Indicators 

I student Short-term Strategic Information 

\ Indicator | Demand Demand | Goals Use in Needs Assessment Comments Sources 

Overall Labor Market Indicators 
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Table 2. Specific Needs Assessment Indicators ' 

Student Short-term Strategic information 

_ _ Indicator Demand Demand Goals Use in Needs Assessment Comments Sources 
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Table 2. Specific Needs Assessment Indicators 

Student Short-term Strategic Information 

__J indicator Demand Demand Goals Use in Needs Assessment Comments Sources 

Occupational Demand Indicators, Continued 
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Table 2. Specific Needs Assessment Indicators 

Student Short-term Strategic information 

Indicator Demand Demand Goafs Use in Needs Assessment Comments Sources 

student Trainihg Demand Indicators 
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Selecting Training Needs Indicators: Step by Step 



Step 1. Identify and Document Available Data. 

In this step, you will review the list of indicators and find out what data are available to 
you. The result will be a list of indicators for which data are currently available. 

This step will require the assistance of individuals Determining Information I 

who are expert in the various data sources, Requirements: > ■ 

particularly individuals from your country’s : : ; v r ' 

population and employment statistics agencies, and I 

someone who is familiar with the available data ’ oa'j " ' 1 

collected on education and training programs. I 

2. Decide. what. additional.data . ^ 

In this review, you should focus on the list of should be collected. I- 

indicators that are appropriate for the decision - ' ' '• ' ' 

drivers that were confirmed in Phase 2, Establishing the Policy Framework, Step 1, 

Review the Goals of Vocational Education and Training. 

To assist you in this step, a worksheet is provided at the end of this section that lists all 
the indicators from Table 2, along with the relevant decision driver for each indicator. 

The last three columns of the worksheet provide an area for you to record whether data 
are available for this indicator, what the data source is, and, if data are not available, 
whether you recommend collecting it. 

In the data identification process you should assess the quality of the available 
information, especially its currency and accuracy. You ^vill need to decide whether the 
available data are of good enough quality for use in the needs assessment. Remember 
that making high-stakes decisions may require higher quality information. 

TIP; 

In addition to the worksheet, you should keep notes on each of the indicators and 
data sources, recording the following types of details: 

■ Who provided information about the available data sources (name, position, 
telephone number, etc.), 

■ Definitions, concepts, and data collection methods, 

■ The media in which the data are available (paper, type of electronic file), 

■ When the data were collected and their reference period, 

■ The geographic area covered, 

■ The classification system used and the level of detail provided in the data, such 
as occupational or industrial detail, 

BEST COPY AVAIUBLE 
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2. Decide. what. additional data . ^ 
should be:Collectect.,. 



Assessing Vocational-Technical Training Needs 



43 



Page 41 



■ Your assessment of data quality (currency, accuracy, whether the definitions and 
level of detail are appropriate to the decision being made, whether the coverage 
is appropriate), 

■ Other details you think are important. 



Step 2. Decide What Additional Information Should Be Collected. 

Completing Step 1 will tell you what information presently is available. You will need to 
decide whether the available information is sufficient or whether additional information 
must be collected. 

In making this decision, you should consider the nature of the decision to be made. 
Generally, the more critical the decision, the more indicators you should have and the 
higher their quality should be. Also, the existing indicators may conflict with each other 
and suggest different conclusions. In these cases, additional information is probably 
needed to reach a sound decision. 

Data collection methods are described in Phase 6. In making decisions about what 
specific information to collect, you should consider several factors: 

■ Difficulty and cost of collecting the additional data, 

■ Time required to collect the data, 

■ Expertise needed to assist in the data collection and whether the expertise is 
available, and 

■ Benefit expected in comparison with the cost and time needed. 



TIP; 



Making the decision about data collection is an excellent opportunity to involve the 
Stakeholder Group. You can present the results of Step 1 to the Stakeholder Group, 
informing them of the available data, and discuss recommendations on what should be 
collected, along with information about difficulty, costs, time, expertise needed, and 
benefits. 
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Phase 5 Worksheet: Identiflcation of Data Available and Indicators Recommended for Data Collection 
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Phase 5 Worksheet: Identification of Data AvaUable and Indicators Recommended for Data Collection 
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PHASE 6. GATHERING ADDITIONAL INFORMATION 



Three general methods are available for gathering additional information: using 
administrative records, conducting surveys, and carrying out group processes. 

Data from administrative records have the advantage of using information that already is 
being collected for another purpose. Problems can arise in getting access to the data and 
with the timeliness and coverage of the data. 

Surveys are formal data collection processes and, if well designed and properly 
conducted, can provide high-quality information for decision-making. Surveys can be 
costly and take time to develop and conduct. 

Group processes, including methods such as focus groups and Delphi techniques, are 
useful in gathering qualitative information such as the expectations, opinions, and 
perspectives of employers, students, and other target audiences. Information from group 
processes may be less representative than information from surveys but may provide in- 
depth qualitative information that cannot be obtained from surveys. Group methods may 
also be used to develop consensus on topics such as goals. 

Each of the data collection methods is discussed briefly below. 

Administrative Data 

Administrative data consist of information extracted from existing records such as 
employer tax reports and school records. Administrative data can be invaluable, as they 
may provide the “universe” list of target groups such as employers and provide a less 
expensive data source than surveys. 

A number of pitfalls exist when using administrative data, however. 

■ The cooperation of the “owner” of the data must be obtained. Creating 
administrative databases requires access to the administrative records from 
which the data items are extracted, and these records are held by the 
organization responsible for the process that generates the records, such as a tax 
agency or an individual school. Often, these organizations do not see providing 
data as their mission, or they may have legal restrictions on access to the 
records. 

■ Administrative data may not be timely. Timeliness depends partly on the 
schedule associated with the administrative process, such as tax filing schedules. 
Extra time may be required to get and process the administrative records, as 
compared with operating a survey, especially if the records are not in electronic 
format. 

■ The data may not be in a convenient or even a usable format. For example, it 
may only exist on paper records that may not be removed from the site. The 
time, effort, and cost of putting the data into usable form should be evaluated. 

■ The data must be coded by, for example, adding industrial classification codes to 
each employer record or geographic area codes to each address. 
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Administrative data files usually have missing reports or late reports, for 
example, from employers who fail to file tax reports or file late. 

The concepts, definitions, and coverage of the administrative data must be 
examined to get a clear understanding of what the data represent. For example, 
an employer database derived from tax records will cover only employers who 
are subject to the relevant tax requirements. This may exclude certain types of 
employers, such as those in the agricultural sector or the self-employed. 



Surveys 

Surveys may be used to collect data from any of several different target audiences: 
employers, individuals surveyed at their residence (household surveys), individuals 
surveyed at their workplace (employee surveys), individuals surveyed at school (student 
surveys), and schools surveyed to gather institution and program data (school surveys). 

Employer Surveys. Employer surveys are a common needs assessment tool, even where 
comprehensive labor market data are available. Employer surveys are used to provide 
information not included in the labor market data and sometimes to confirm or update the 
labor market data. Where labor market data are weak or not available, employer surveys 
become critical. 

A side benefit of employer surveys is the message they inherently give to employers: 
that their input is important and that the vocational-technical training programs are trying 
to meet their needs. 

Employer surveys must be carefully designed, however, maintaining a focus on how the 
results will be used. Typically response rates in such surveys are relatively low, and it 
can be tempting to ask too many questions, reducing employer response rates even 
further. 

Household Surveys. Household surveys are usually the best method of collecting 
demographic information and other data about individuals that cannot be collected from 
employers. Household surveys often include questions on topics that are also covered in 
employer surveys, such as the industry or occupation of the respondent’s employment. 
Household data on these topics permits analysis by demographic characteristics, which 
usually is usually not possible with employer survey data. An example is the analysis of 
occupational employment by gender and age of the workers. Age and gender are easily 
collected in household surveys but may be difficult to collect from employers, who may 
not have this information easily accessible or may be reluctant to provide information on 
the personal characteristics of their workforce. 

Household surveys also capture information about individuals who are not employed, 
which obviously cannot be done with employer surveys. 

Employee Surveys. This method collects information from individuals who are contacted 
at their place of work and is quite useful for gathering in-depth information about specific 
occupations or industries. These surveys can be quite difficult to conduct, however, as 
they require the cooperation of both the employer and the worker. Incumbent worker 
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surveys may require methods to protect the privacy of the individual worker’s responses 
from access by the employer. 

As will be noted in the section of this Guide on company needs assessment, incumbent 
worker surveys are important in assessing training needs within the company. 

Student Surveys. These surveys collect information from students who are contacted at 
their school. The cooperation of the school as well as the student is needed. Student 
surveys, such as tracer studies, are often conducted by schools themselves. 

School Surveys. School surveys may be used to gather information about the institution 
and its services, such as enrollment, programs offered, number of students completing the 
programs and/or receiving diplomas or other credentials, number of teachers, financial 
information, information on facilities, and other data. School surveys should not 
duplicate collection of information that is required in administrative reports. 

Sample Survey or a Census? Surveys may be conducted by contacting a sample of 
respondents or contacting all potential respondents through a census. The choice 
between a sample survey and a census is a critical one. Unless there is a specific reason 
to conduct a census, such as the need for a baseline complete count that is provided by 
population censuses, the sample survey is the preferred choice. 

The guiding principle is to use the method that will achieve results that are 
representative of the overall population being studied. In employer surveys, for 
example, results are needed that are representative of all employers in the area or all 
employers in a particular industrial sector. 

Conducting a census might sound like the best approach to those unfamiliar with basic 
statistical principles. It may be argued that since a census probably will provide a greater 
number of responses than a sample survey, census data are just as good as (that is. Just as 
accurate and representative) or better than sample survey results. Unless a very high 
response rate is achieved, this argument is in error. 

As any standard statistical reference will discuss, the results from a census represent only 
the characteristics of the employers or households that actually respond to the survey. 

The results cannot represent the characteristics of the entire population unless, as the term 
“census” implies, nearly all of the population is included in the count. 

A census also is more expensive to conduct that a sample survey, since all members of 
the target group (employers, households, etc.) must be contacted. 

Representative data usually can be more easily and cheaply collected with a well- 
designed sample survey. Sample surveys provide data for which measures of accuracy 
(sampling error) can be calculated. The cost is less than a census, as many fewer contacts 
are needed. 

For gathering data on very small populations, sample designs might include virtually all 
members of the population, especially if detailed presentations of the data are planned. In 
such cases, conducting a census instead of a sample survey may be appropriate. 
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Response Rates. Response rates are critical to the success or failure of a survey. Unless 
the response rate is at an acceptable level, the survey results will be useless. For sample 
surveys, selecting a target response rate is part of the sample design process. The lower 
the target response rate, the larger the sample must be to achieve results with acceptable 
accuracy and the more difficult and expensive the survey will be to conduct. For census 
surveys, very high response rates are required to achieve representative results, as 
discussed earlier. 

What are typical response rates? The response rates that can be expected vary 
tremendously for different types of surveys, and according to how the survey is designed 
and conducted. Dillman notes that mail surveys he reviewed during preparation of his 
Total Design Method averaged about 77 percent, and that telephone surveys produced 
somewhat higher response rates. The standard for federal government sponsored 
surveys in the United States is 80 percent. The response rate for the 2000 Occupational 
Employment Statistics Survey, an employer survey providing employment and wage data 
by occupation, was 78 percent. 

Achieving these high response rates requires skilled and experienced staff and well- 
designed and tested survey procedures. Most surveys produce far lower response rates, 
especially if they are conducted by organizations without experience. 

Good survey design and implementation can increase response rates. Dillman 
recommends a variety of ways to improve response rates, based on the theory of surveys 
as “social exchange.” He argues that individuals will be motivated to respond by three 
factors: the reward they expect to gain, the cost to them of responding, and their level of 
trust that the rewards will outweigh the costs. “Rewards” in this context are not 
necessarily economic rewards but may be intangible rewards such as demonstrations of 
respect, making a contribution to improvement of one’s community, or feeling valued 
because one’s advice has been requested. “Costs” may include the literal cost of the time 
and effort required to respond, as well as the cost of disclosing personal or business 
information, and other social costs." 

Based on the theory of social exchange, Dillman has identified and tested a number of 
basic methods to increasing response rates: 

■ Increasing rewards by using language in cover letters that shows the value 
placed on the respondent and their advice, saying “thank you” in advance and 
follow-up contacts, indicating how the survey will be used in making decisions 
that will benefit the community, and making the questionnaire itself interesting 
to respondents. 

■ Reducing the cost to respondents by making the questionnaire short and easy to 
answer, providing return postage for mail surveys, using wording that asks for 
the respondent’s assistance rather than authoritatively demanding a response, 
minimizing questions about personal or proprietary information, and avoiding 
embarrassing the respondent by, for example, requiring complex reading or 
writing that he or she may not be able to perform. 
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Increasing trust by providing tokens of appreciation in advance such as an item 
or a small amount of money, having the survey sponsored by a legitimate 
authority, emphasizing the importance of the survey. 



Sources of Error in Survey Data. Survey data are subject to various sources of error, 
which usually can be managed by good survey design and execution. Four types of error 
are important: sampling error, coverage error, measurement error, and nonresponse error. 

Sampling error occurs because, in sample surveys, some respondents are selected to 
represent all respondents. Sampling error can be explicitly measured by calculating the 
“margin of error” or “standard error of the estimate” when the survey results are 
tabulated.*^ Sampling error can be reduced through good sample design and obtaining 
high response rates. NOTE; Sampling error does not apply to census surveys, and the 
“margin of error” for a census cannot be produced. 

Coverage error occurs because the population being studied is not completely covered by 
the survey’s “universe list”. Some members of the population therefore do not have the 
opportunity to be represented. For example, a telephone survey of a population that 
includes many individuals without a telephone would have high coverage errors. 
Coverage error can be managed by obtaining a universe list that is as complete as 
possible, and using appropriate data collection methods. 

Measurement error occurs when the survey questions are not well designed. Poor 
question wording may result in respondents skipping questions or providing inaccurate 
responses to questions they do not understand. Measurement error may also result from 
questions that, while clear, do not accurately reflect the information the survey is 
intended to capture. Measurement error can be reduced by good questionnaire design, 
including the critical step of testing the questionnaire with potential respondents. 

Nonresponse error occurs because the respondents to the survey are different from those 
who did not respond in ways that affect how they respond to the questions. For example, 
if the respondents in an employer survey on wages are disproportionately large 
businesses, which are more likely to pay higher wages than small businesses, the 
resulting wage data will have a bias toward higher wages. Nonresponse error can be 
managed through good sample design, achieving good response rates, and perhaps by 
statistical adjustments for nonresponse bias. Analysis of the characteristics of 
nonrespondents in comparison with respondents is very useful in understanding 
nonresponse error. NOTE: Census surveys that do not achieve high response rates are 
particularly subject to nonresponse error. 



Other Resources About Surveys. A wide range of resources is available to provide 
additional background information about surveys and how to conduct them. Two may be 
particularly helpfiil: 

■ The American Statistical Association publishes several pamphlets on survey 
topics that provide good background for a non-technical reader. Topics covered 
include What Is a Survey, How To Plan a Survey, How To Collect Survey Data, 
Judging The Quality of a Survey, Mail Surveys, Telephone Surveys, Margin of 
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Error, and Designing a Questionnaire. These pamphlets are available on-line at 
http://www.amstat.org/sections/srms/whatsurvev.html. 

Don Dillman’s book, Mail and Internet Surveys: the Tailored Design Method 
(2"** edition, John Wiley and Sons, New York, 2000), is a practical guide to 
designing and conducting surveys. 



Group Processes 

Surveys and administrative records are excellent tools for collecting quantitative data. 
However, qualitative information, such as the opinions and attitudes, may also be 
valuable in the needs assessment process. Group processes provide the opportunity for 
face-to-face interactive communication with members of important groups who have 
knowledge about and a stake in vocational education and training, such as employers, 
students, parents, and educators. Group processes may also be used with the Stakeholder 
Group in developing ideas and goals for the needs assessment. 

Group processes have the advantage of providing a wide range of information quickly 
and allowing unanticipated topics to be identified and explored. 

Group processes also have some disadvantages. They do not produce representative 
information, as they reflect only the views of the participants, which cannot be 
generalized to the entire population. Also the results are influenced by the skill and 
motivation of the group facilitator. Because of these limitations, information from group 
processes should not be the only source of information in the needs assessment. Group 
process results should be used along with quantitative data from existing data sources, 
surveys, or administrative data. 

While a variety of group processes are available, three may be particularly relevant to 
training needs assessment: the Nominal Group Technique, the modified Delphi 
technique, and focus groups. 

Nominal Group Technique. The Nominal Group Technique (NGT) is a structured 
process to identify and rank problems or issues affecting the group, or managing 
participation in planning, performance measurement, and other processes. NGT is used 
to gather generate a large number of individual views and ideas on a particular topic and 
to achieve consensus on a list of priorities, measures or actions. 

NGT sessions should include 6 to 10 invited participants and a skilled facilitator. Each 
session should be limited to discussing one or two questions that have been prepared in 
advance. If more than one or two questions need to be examined, several NGT sessions, 
each with different participants, should be held. 

Modified Delphi Technique. The Delphi technique is characterized by the iterative use of 
a mail survey over time with the same panel of respondents. Between each iteration, the 
extent of agreement or disagreement among the panel members is identified, and this 
information is shared in the next iteration. The process thus helps build consensus. The 
“modified” or group Delphi technique adapts the procedure for use in a face-to-face 
group setting. The Modified Delphi technique is often used to develop forecasts of future 
trends. 
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Focus Groups. Focus Groups are widely used to interview a small group (8 to 12 people) 
to obtain their in-depth views. A skilled facilitator is required. Focus Groups provide an 
opportunity for probing and follow-up questioning by the facilitator to explore responses, 
and the opportunity for interaction among the participants can generate additional 
insights. Unlike NGT and the Modified Delphi Technique, the focus group process does 
not attempt to obtain consensus. 

Resources about Group Processes. A number of resources are available to guide the 
selection and use of group processes. Especially helpful examples are: 

■ Planning and Conducting Needs Assessments: A Practical Guide, by Belle Ruth 
Witkin and James W. Altschuld, Sage Publications, Thousand Oaks, California, 
1995. 

■ The American Statistical Association’s pamphlet What are Focus Groups?, 
available on-line at http://www.amstat.org/sections/srms/whatsurvev.html. 

■ The University of British Columbia’s on-line list of resources on qualitative 
group methods, available at 

http://www.amstat.org/sections/srms/whatsurvey.html. 



Gathering Additional Information through Surveys: Step by Step 

Step 1. Get Access to Survey Expertise. 

Conducting surveys is a specialized field. Unless 
you have expertise in designing and operating a 
survey, including expertise in statistics, you should 
obtain the assistance of individuals or an 
organization with the needed skill and experience. 

This type of experience is often available from 
statistical agencies, universities, private vendors, and 
international organizations. 

Capacity for conducting surveys includes: 

■ Staff with the statistical expertise to design 
and select a sample, develop and test the 
survey questionnaire, and calculate final 
survey results based on the responses 
received, 

■ Staff with experience in managing and 
conducting the survey operation, including 
printing, mailing or telephone operations, 
tracking responses received, and following 
up on nonrespondents, and 
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Computer resources, including equipment and software programs for statistical 
tasks and survey operations. 



Step 2. Deflne the Information to be Collected. 

In Phase 5, you have identified the type of information needed by comparing the list of 
indicators to available data sources. In data collection, you need to define specifically 
what you want to collect and from whom. 

For example, you may have determined that you need an indicator of current job 
vacancies by occupation (item 12 on the Phase 5 worksheet). To collect this data, you 
need to decide exactly what you mean by “current job vacancies by occupation.” 

■ For what time period? 

■ How will “job vacancy” be defined so survey respondents will provide accurate 
responses? 

■ How will the vacancies be classified by occupation? Will you provide the 
respondent with a list of occupations, or will you classify job titles provided by 
the respondents into a standard occupational classification? 

■ Do you need to know anything about the job vacancies besides occupation, such 
as the wage being offered or how long the vacancy has been in existence? 

In defining the information to be collected, you must keep at the forefront how the data 
will be used in the decision-making process. 

Step 3. Define and Identify the Survey Universe. 

In this step you will determine from whom you will collect the data. Is it to be collected 
from employers, households, incumbent workers, or students? Should there be any limits 
on the universe, for example, only employers in certain industries or only employers with 
more than a certain number of workers? 

Once you know the target audience for the survey, you will need to identify a list of 
specific potential respondents that will constitute your survey “universe.” For employer 
surveys, the “universe” is the complete list of employers, including any desired limits 
such as industry or number of workers. For household surveys, the universe is a list of 
households, and so forth. Obtaining a good universe list is key to reducing coverage 
error in the survey results. 

TIP: 



Be sure to examine the universe list to be sure you understand what it includes and 
how it was derived. 

Dillman suggests the following questions to evaluate universe lists*'*: 

■ Does the list contain everyone in the survey population? 
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“ Does the list include nnmes people or businesses that are not in the survey 
population? 

■ How is the list maintained and updated? 

■ Are sample units included on the list more than once? 

■ Does the list contain other information that can be used to improve the survey? 
You should also ask: How recent is the list? 

Step 4. Determine the Data Collection Method. 



You will need to decide whether the survey will be conducted by mail, by personal 
interview, by telephone, or by Internet. This choice will be driven by the nature of the 
information to be collected, who the information will be collected from (employers, 
households, etc.), costs, the availability of information needed to conduct the survey 
(mailing addresses, telephone numbers), and capacity to implement the alternative 
collection methods. 

TIP: 

The University of Florida Cooperative Extension Service Circular PE-21 provides 

a useful guide on selecting a data collection technique. The circular is available 
on-line at http://edis.ifas.ufl.edu/PD016. 



Step 5. Develop the Analysis Plan. 

Before developing the actual survey instrument and sample, you should decide how you 
will analyze and present the resulting data. One good way to do this is to lay out table 
formats you will use for presenting the results. This exercise is important, because the 
survey can then be developed to provide the information required in the tables. 

In an example, you may decide that you will need to analyze job vacancy data from your 
survey by specific industry and geographic area. This decision will affect the sample 
design, as a larger sample will be needed to provide accurate data by industry and 
geographic area that would be needed just to provide an estimate of the total number of 
job vacancies. 

Step 6. Design the Survey Sample. 

In doing this step, you will first need to decide whether your survey will be a census or a 
sample survey. As discussed earlier, a sample survey is usually the preferred choice. 

Your statistical expert should develop the sample design, taking into account the analysis 
plan, the expected survey response rate, and the level of accuracy desired in the resulting 
data. The design might also include features to adjust for nonresponse biases, that is. 
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biases in the resulting data that are caused by differences in the characteristics of 
respondents in comparison with nonrespondents. 

Step 7. Develop the Survey. 

In this step you will develop and test the survey questionnaire, including the wording of 
the questions and the exact sequence and format. You will also need to develop a letter 
or postcard to be sent out in advance of the survey and, for mail surveys, a cover letter to 
be mailed along with the questionnaire. Finally, you should develop the procedure, 
schedule, and materials for follow-up contacts with nonrespondents, such as reminder 
letters or a script for telephone follow-up. 

A critical part of this step is to test the questionnaire. You will need to find out how 
long it will take the respondents to complete the questions, whether they will understand 
the questions, and whether they will be willing and able to provide an accurate response. 

TIPS: 

Keep It Simple! The longer and more difficult the questionnaire, the lower the 
response rate will be. The questionnaire should cover the necessary topics and use 
a clear, easy-to-follow format but should not include extraneous questions or long, 
complicated text. 

Use American Statistical Association’s pamphlet Designing a Questionnaire, 
which contains a number of simple guidelines to good questionnaire design.*^ 

Test the questionnaire with several actual potential respondents. Check the 
American Statistical Association’s pamphlet How to Conduct Pretests for 
guidelines. 

During the test, time how long it takes respondents to complete the survey. Then, in the 
survey cover letter or instructions, give respondents information on how long it is likely 
to take them to complete their response. 

Be sure to plan prompt follow-up with nonrespondents to encourage response. For mail 
surveys, this should include sending another copy of the questionnaire. Testing has 
shown that nonresponse follow-up can increase response rates by 20 to 40 percentage 
points and that several follow-ups may be needed.* 



Step 8. Gain Respondent Interest and Cooperation. 

You may be able to improve response rates by doing outreach to potential respondents to 
let them know about the survey and its purpose. If they understand the benefits of the 
survey, they will be more likely to respond. 
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TIPS: 



Involve your Stakeholder Group in this outreach to potential respondents! 



Have the cover letter signed by a known and respected authority. Also include the 
name and telephone number of an individual who can be contacted for more 
information about the survey. 

Step 9. Conduct the Survey. 

Once the sample has been designed, the survey instrument is developed and tested, 
procedures and materials are prepared, and outreach has been conducted, your survey 
operations experts should be ready to conduct the survey. The major steps include: 

■ Drawing the sample, using the universe list and the sample design to produce a 
list of respondents who will be contacted, 

■ Refining the respondent list to be sure complete and accurate information is 
available for all employers or individuals on the list, such as mailing address (for 
mail surveys) or telephone numbers (for telephone and personal interviews), 

■ Printing the quantity of questionnaires, letters and/or postcards needed, 
including the quantity needed for follow-up contacts, 

■ Sending cover letters or advance post cards to go out with or ahead of the 
survey, and reminder notices to nonrespondents, 

■ Conducting the data collection, including doing the mailing, interviews, or 
telephone calls, monitoring the collection to identify respondents and 
nonrespondents, and making the follow-up contacts with nonrespondents, 

■ Following up with nonrespondents, such as reminder letters or postcards, 
contacting nonrespondents by telephone, and doing follow-up mailings of mail 
surveys, 

■ Entering the data from survey respondents, 

■ Editing and evaluating the data, including checking for internally consistent 
responses and deciding whether the response rate is high enough to provide data 
of acceptable accuracy, and 

■ Tabulating the survey results, including calculating the statistical estimates and 
creating tables using the table formats designed in Step 5. 



TIP: 

Be sure that survey staff are trained in how to conduct the survey! Training should 
cover the purpose of the survey, the use of the information it will provide, the 
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content of the questionnaire, the survey schedule, and the procedures to be used. For 
telephone surveys, interviewing techniques and practicing the telephone interview should 
be covered in the training. 



Step 10. Analyze and Distribute the Results. 

In this step, you will use the survey results to develop an understanding of the indicator 
the survey was intended to provide. For example, in a survey of current job vacancies by 
occupation, you might prepare a description of the number and type of job vacancies 
identified in the survey and discuss what this information means for the vocational 
education and training program decisions that are being 



A written report of the survey results should be 
prepared that describe the results and your analysis, 
as well as describing how the survey was designed 
and conducted, what the response rates were, and the 
quality of the resulting information. This report 
should be distributed to the appropriate audiences. 



Gathering Additional Information through 
Group Processes: Step by Step 

Regardless of the particular group process method 
used, some basic planning and execution steps must 
be carried out. These are listed below, following the 
structure identified by Witkin and Altschuld . For 
more information on specific group process methods, consult the sources listed in the 
References section of the Guide. 

Step 1. Establish clear purposes for the group process. 

Be sure the purpose and outcomes for the group process are clearly defined, so 
participants will know why their participation is important and what is expected of them 
and so the facilitator can work effectively. The clear statement of purpose will also be 
important in selecting the appropriate group process method and developing the 
procedures. 

Step 2. Select the group process. 

Review the alternative group processes - Nominal Group Technique, Focus Groups, and 
Modified Delphi Technique - and decide process best fits the purpose identified in Step 
1. Your choice will also be affected by the relative difficulty of the different methods 
and the availability of facilitators who are skilled in the different methods. 

BEST COPY AVAILABLE q 
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made. 

Gathering Additional 
Information through Group 
Processes: 

Step by Step 

1 . Establish clear purpose. 

2. Select the group process. 

3. Identify a skilled facilitator. 

4. Develop procedures and 
arrangements. 

5. Invite the participants, 

6. Conduct the process. 

7. Analyze the results. 

I 8 . Communicate the results. 



Step 3. Identify a skilled facilitator. 



Get the assistance of an individual who has been trained in the group process method. 
Note that it is not essential that the facilitator have in-depth knowledge of the subject 
matter that ’will be discussed. It is important that the facilitator not try to influence the 
group’s thinking and not interject his or her personal opinions in the discussion. The 
process is to get the group’s thinking, not the facilitator’s. 

Step 4. Develop the procedures and arrangements. 

In this step, you will need to develop the specific materials needed for the process, such 
as questions for the group’s response. You will also need to make arrangements for a 
place for the group session, materials needed in the room (such as flip charts and 
markers), and prepare the agenda. 

Make sure the procedures include a way of recording the group’s ideas and consensus. 
This might include providing someone to serve as recorder or observer, as well as 
capturing notes and flip charts generated during the session. Recording the results will 
allow you to refer to them accurately when decisions are being made using the group’s 
input. 

TIPS: 



The group may be more effective if the group session is held at a place away from 
the participants’ usual place of work. This will help them avoid distractions. 

Make sure the meeting room is comfortable! And provide breaks and 
refreshments. 

Step 5. Invite the participants. 

The participants should be individuals with knowledge of the topic being discussed and 
have an interest in the outcome. The number of participants should be appropriate for the 
technique selected. 

Be careful about the mix of people invited. For example, in the NGT, a cross-section of 
people from different constituencies can effectively participate, while Focus Groups work 
best if the participants share similar backgrounds. 

Avoid inviting supervisors and subordinates from the same organization to the same work 
group. They may not feel free to express their views in front of one another. 

Step 6. Conduct the process. 

On the planned schedule, the facilitator will conduct the group activity.. 
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Step 7. Analyze the results. 

Immediately after the group’s work is completed, prepare a summary of the results, 
including any statements of consensus, findings, or recommendations generated by the 
group. Provide this summary and a “thank you” to the group participants. 

Step 8. Communicate the results. 

Provide the results to the decision-makers, along with other information being used in 
their deliberations. 
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PHASE 7. ANALYSIS AND PRESENTATION 



With the conclusion of Phases 5 and 6, you will have available all the information to 
complete the needs assessment. What remains to be done is to analyze the information to 
identify the specific training needs. You also should present this information to the 
vocational and technical education decision-makers. 



The specific analysis to be done will depend to a great 
extent on the information available from Phases 5 and 6. 
Regardless of the analytical method used, however, the 
analysis should lead to identification of relative need for 
various training programs, either in the form of lists of 
programs or lists of occupations in demand for which 
programs should be provided. 

The results of the needs assessment should be put into a 
written report, and an oral presentation should be 
developed for any appropriate audiences, such as the 
Stakeholder Group and the program decision-makers. 
The written report is essential so the results can be 
accessed by individuals who may not be available for 
presentations, as well as to provide a record to document 
decision-making. 



. 

Analysis and ! 

Presentation: j 

Step by Step | 

ii 

' 1 . Assemble the 

information. 

2. Decide on analysis 
method. 

3. Conduct the analysis. 

4. Prepare written report. | 

5. Prepare and give o.ml ii 

presentations, I 



Analysis and Presentation: Step by Step 

Step 1. Assemble the available information. 

Put the data and information from focus groups or other qualitative activities into a 
format that permits analysis. This might be in the form of databases that permit querying 
the data, spreadsheets for ranking and comparison, or other forms. 

Step 2. Decide on analysis method. 

You will need to choose the analysis methods that best fit the information available and 
address the policy questions. Examples of methods include comparisons, rankings, 
computing scores that represent several indicators, and writing narrative analyses for 
different occupations or programs. 

Step 3. Conduct the analysis. 

In this step, you will apply the analysis methods to the information, identify patterns or 
key points, and develop conclusions or recommendations. 
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Step 4. Prepare written report 

In this step, you will write the needs assessment report. The report should cover the 
following topics, at a minimum: 

■ A description of the policy context in which the needs assessment is being 
conducted and the vocational-technical education goals or drivers being 
addressed. 

■ The scope of vocational-technical education covered by the needs assessment. 

■ The target geographic area. 

■ A summary of the Economic Review. 

■ A summary of the methods used, such as surveys or group processes. 

■ The results, including the analysis leading up to the results and a list of the 
programs or occupations identified as training needs. 

■ Documentation about the data sources and methods used, perhaps shown as an 
appendix. 



TIP: 

Be sure the needs assessment report acknowledges the role and contributions of the 
Sponsor and the Stakeholder Group. 



Step 5. Prepare and give presentations. 

In preparing the oral presentation, keep in mind that usually only the highlights can be 
covered in a presentation. There will not be time for covering all of the material 
contained in the written report. Also, be sure to tailor the presentation to your audience. 
How much do they already know about the needs assessment project? Do they want to 
know the details of the study or just the highlights and recommendations? 
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PHASE 8. PUTTING NEEDS ASSESSMENT RESULTS TO WORK 



Once decisions are made on the training programs to be offered, several additional 
activities are needed to make full use of the needs assessment results. These are directed 
toward providing information for developing the training curriculum, and keeping the 
curriculum up to date with developments in the work place and the labor market. For 
these programs to be effective, data about the programs must be collected, and career 
information must also be provided to current and prospective students. 



Putting Needs Assessment Results to Work: Step by Step 



Step 1. Develop and update curriculum. 



The needs assessment probably resulted in decisions about 
which training will be offered. Translating a list of 
occupations or programs into training curriculum 
requires activities to identify the specific skills and 
knowledge that must be taught in the program to qualify 
students for work in the occupation. 



occupations or programs for 



Putting Needs 
Assessment Results to 
Work: 

Step by Step 



Job analysis is the tool for identifying this skill and 
knowledge information. As discussed in the following 
section on needs assessment within a company, several 
job analysis methods are available. One of the most 
often used methods is DACUM. 



1 . Develop and update 
curriculum. 

2. Monitor the labor 
market. 

! 3. Provide career 
information 



Once a program curriculum has been prepared and is put j Collect ana analyze 

into use, it is important to keep it up to date. Skill j program information 

requirements can change rapidly, and the knowledge 

workers must have to perform their jobs may also change. It is critical, therefore, to keep 
employers involved in reviewing and advising on program curriculum. The creation of 
employer advisory committees, for example, will help tie curriculum to current practice 
in businesses that may hire program graduates. The References section notes resources 
about employer advisory committees. 



Step 2. Monitor the labor market. 

The economy and the labor market are dynamic. Change is occurring all the time and 
can be abrupt and unexpected. Thus, it is important to monitor the labor market to spot 
changes that could affect decisions about vocational and technical education program 
offerings. These may be specific events or major changes in labor market trends or 
smaller changes that call for continuous adjustment of training investments. 
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Step 3. Provide career information. 

In a vocational technical education system where student choice is the primary driver of 
decisions on program offerings, providing career information to students is vital. 

Students need to be know about the careers for which vocational-technical programs will 
prepare them, how well the work in these occupations fits their interests and goals, what 
the job and earnings prospects are like, and what additional education and training they 
may need. They need to be equipped to make informed choices. 

Even in systems where student choice is not the primary driver, career information plays 
an important role. Offering programs to prepare workers for demand occupations needs 
to be accompanied by providing information to current and prospective students about 
how these programs will prepare them for good careers that meet their personal interests 
and goals. Without such information, the programs may fail for lack of student 
enrollment. 

Step 4. Collect and analyze program information. 

Needs assessment is a continuing process that should include regular review of programs 
to track the level of student enrollment, student completion of the programs, and tracer 
studies to identify student outcomes. Such information also is critical for overall 
management of the vocational technical education system. 
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TRAINING NEEDS ASSESSMENT WITHIN A COMPANY 



Part of good human resource management in any business or organization is providing 
training for staff development. Providing the right training - training that is needed and 
will benefit the company as well as the employees - is key. Training must be tailored to 
the business’s strategy and fit within its organizational structure and work processes. 

Needs assessment is the foundation for successful company training. Needs assessment 
will identify which employees should receive training, the type of training they should 
get. The assessment will lead to development of appropriate training curricula. Doing 
solid needs assessment will help the company avoid wasting its resources - money, time, 
and physical resources - on training that is not appropriate. 

“Company training” as referred to here includes training the company provides to its 
employees directly, as well as employee training sponsored by the company but provided 
by others, such as schools, technical colleges, and private providers. 

In general, needs assessment is a process of identifying the skill and knowledge 
requirements of the work, assessing the skill and knowledge levels of the experienced 
employees or newly hired employees and developing training to address the gap between 
requirements and employee capabilities. The result should be a set of objectives 
describing the purpose of the training and the competencies employees should have when 
training is completed. 



Reasons for Providing Company Training 

The initial issue that must be examined in assessing training needs is. What is the nature 
of the business problem that training is to address! Companies provide employee 
training for a number of different reasons, and each reason can lead to a different type of 
needs assessment. Typical reasons for company training are discussed below. 

Training new employees in company-specific procedures, tools, and processes. New 
employees often need training in operating the specific equipment used by the company, 
in company procedures and standards, and in other company-specific topics they would 
not be expected to know prior to their employment. Training should be based on an 
analysis of the job content and the expected capabilities of new employees. Planning fqr 
new employee training must be coordinated with recruitment and hiring, so employees 
hired have the expected knowledge and skills. 

Training new and current employees to address basic or general skill deficiencies. 
Companies may find that the workers available in their labor markets do not possess 
basic skills (reading, mathematics) or general skills and knowledge. Ideally, potential 
employees gain basic and general skills and knowledge through their education 
experience. Where this does not happen, companies may decide to provide training. 



O 

ERIC 



Assessing Vocational-Technical Training Needs 



66 



Page 65 



Training needs should be identified based on analysis of the basic and general skills and 
knowledge required by the jobs, and assessment of the skills of individual workers. 

Training to address performance problems of specific employees or groups of employees. 
Training sometimes is needed because employees are not correctly carrying out work 
tasks. When a performance problem is identified, it should be analyzed carefully to 
determine if it is caused by lack of job skill or knowledge. These causes can be 
addressed by training, while other causes, such as poorly designed work processes, are 
not training problems. Task analysis and individual worker skill assessments may be 
useful in determining what training is needed. 

Training current employees in use of new procedures, equipment, and processes. New 
ways of doing work, such as new computer software, new safety procedures, or a new 
machine tool, usually require employee training. Such training is designed based on the 
requirements of the new procedures, equipment, or processes. In this situation, task 
analysis may be used. Individual worker assessment probably is not relevant, as the 
workers would not be expected to have the newly required skills and knowledge. 

Training to develop current employees for promotion and advancement. Training for 
advancement may be more complex than training provided for other reasons. The 
requirements of jobs into which advancement is anticipated must be identified. 
Assessment of the ability of current employees to perform these jobs is complicated by 
the fact that they do not currently hold these jobs and have had no opportunity to 
demonstrate their abilities. Special assessment techniques such as assessment centers or 
tests for supervisory activities may be used. 



Three Levels of Company Needs Assessment 

Fisher, Schoenfeldt, and Shaw*^ identify three levels of needs assessment; organizational 
analysis, task analysis, and individual assessment. 

Organizational analysis looks at training within the context of the organization, 
responding to question such as: 

■ How does the training fit in with the organization’s future plans and goals? 

■ Where in the organization is training needed? 

■ Which units should be trained first? 

■ Will this training adversely affect people or units not being trained? 

■ Which training programs should have priority? 

Job and task analysis is the process of identifying duties of the jobs to be involved in 
training, and the knowledge, skills, and abilities needed to perform these duties. A 
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variety of task analysis methods are available, but task inventory and critical incident 
methods are particularly useful. 

Individual assessment determines which employees should be trained and their current 
levels of skill and knowledge. This may include use of assessment methods and should 
pay attention to the workers’ basic skills as well as the job-specific tasks. The workers 
may be unable to benefit from training for job-specific tasks if they do not have the 
necessary level of reading or math skills, for example. 

Once these three levels of assessment have been conducted, the results can be translated 
into training objectives for use in developing the training program. 



Resources for Company Needs Assessment 



Job and Task Analysis Methods . A number of job and task analysis methods are widely 
used in company needs assessments.^® Examples include: 

■ DACUM developing A CurriculUM) uses a focus group of high- 
performing incumbent workers in a facilitated storyboarding process to 
capture their observations on the major duties and related tasks in their 
occupation. The result is a detailed list of the duties and tasks that make 
up their job. As its name implies, DACUM is frequently used in 
curriculum development for training and education programs. 

■ Position Analysis Questionnaire (PAQ) is a structured worker-oriented 
task analysis instrument, using an extensive list of worker-oriented job 
elements. The PAQ reveals the types of interpersonal relationships 
required in task performance, which are often overlooked in task inventory 
analysis. 

■ Common-Metric Questionnaire is a structured job analysis instrument for 
describing work activities. 

More information on these job analysis methods is provided in the References section. 

Individual Assessment Methods . A variety of tools are available for assessing individual 
worker’s performance and skills. Good practice calls for using an assessment tool 
designed to work with the method selected for task analysis, so valid comparisons may be 
made of the worker and the job requirements. These comparisons will guide the 
development of company training efforts. The developers of the PAQ and Common- 
Metric Questionnaire, as well as other task analysis methods, usually also have developed 
assessment tools. 
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and Education Coordinating Board, 2000, at vyww.wa.gov/wtb 

® This section is based on R.A. Wilson, op.cit.. 
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" Dillman, p. 14-23. 

For more information, see the American Statistical Association’s pamphlet “What is a Margin 
of Error,” on-line at http://www.amstat.org/sections/srms/whatsurvev.html . 

This section is based on Planning and Conducting Needs Assessments: A Practical Guide, 

Belle Ruth Witkin and James W. Altschuld, Sage Publications, Thousand Oaks, California, 1995. 

'“Dillman, page 199-200. 

’’ For more information, see the American Statistical Association’s pamphlet “Designing a 
Questionnaire,” on-line at http://www.amstat.org/sections/srms/whatsurvev.html . 

For more information, see the American Statistical Association’s pamphlet “How to Conduct 
Pretesting,” on-line at http : l/www . amstat.org/ sections/s rms/wha t survey . htm l . 

Dillman, page 177. 

Witkin and Altschuld, page 155-159. 

Cynthia D. Fisher, Lyle F. Schoenfeldt, and James B. Shaw, Human Resource Management, 
Houghton Mifflin Company, 1996, p. 362-364. 
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For a brief review of task analysis methods, see ‘Task Analysis Strategies and Practices,” 
Bettina Lankard Brown, 1998, ERIC Clearinghouse Practice Application Brief, available on-line 
at http://ericacve.org/docs/taskanal.htm . 
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